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Abstract 


Background: The integrity of medical research reporting in online news publications is crucial for informed 
healthcare decisions and public health discourse. However, omissions, lack of transparency, and the rapid 
spread of misinformation on digital and social media platforms can lead to an incomplete or inaccurate 
understanding of research findings. This study aims to analyze the fidelity of online news in reporting 
medical research findings, focusing on conflicts of interest, study limitations, statistical data, and research 
conclusions. 


Methods: Fifty randomized controlled trials published in major medical journals and their corresponding 
news reports were evaluated for the inclusion of conflicts of interest, study limitations, and inferential 
statistics in the news reports. The alignment of conclusions was evaluated. A binomial test with a Bonferroni 
correction was used to assess the inclusion rate of these variables against a 90% threshold. 


Results: Conflicts of interest were reported in 10 (20%) of news reports, study limitations in 14 (28%), and 
inferential statistics in 19 (38%). These rates were significantly lower than the 90% threshold (p<0.001). 
Research conclusions aligned in 43 (86%) cases, which was not significantly different from 90% (p=0.230). 
Misaligned conclusions resulted from overstating claims. 


Conclusion: Significant gaps exist in the reporting of critical contextual information in medical news 
articles. Adopting a structured reporting format could enhance the quality and transparency of medical 
research communication. Collaboration among journalists, news organizations, and medical researchers is 
crucial for establishing and promoting best practices, fostering informed public discourse, and better health 
outcomes. 
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Introduction 


The dissemination of medical research findings through online news publications plays a crucial role in 
shaping healthcare decisions and informing public health discourse. However, even in legitimate news 
sources, the reporting of medical research is susceptible to omissions and a lack of transparency, which can 
lead to an incomplete understanding of the research findings [1]. The rise of digital and social media 
platforms has amplified this challenge, as these platforms can rapidly disseminate false claims without 
stringent verification processes, significantly influencing public health perceptions and behaviors. For 
example, health misinformation on social media has been linked to reduced compliance with public health 
guidelines, such as vaccine uptake [2,3], and to the amplification of an infodemic, a mixture of 
misinformation and true information about the origins and alternative cures of disease [4]. 


While considerable research has examined the impact of misinformation and fake news on social media, a 
focused investigation into the reporting practices of legitimate online medical news sources remains 
underexplored. The rapid evolution of digital media platforms has significantly altered the landscape of 
information dissemination, necessitating a closer look at the transparency and completeness of these online 
news outlets. 


This study evaluated online news reports compared to peer-reviewed medical research. Key inclusion criteria 
for readers to reasonably and accurately evaluate the medical news article were assessed. 


Materials And Methods 


The evaluation focused on four critical aspects of reporting: the mention of conflicts of interest (COIs) by 
study authors, the acknowledgment of study limitations, the inclusion of inferential statistical data (either 
p-values or confidence intervals with effect sizes), and the alignment of news report conclusions with those 
of the original research articles. These four variables were all binomial, categorized as yes (included in the 
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news article) or no (not included in the news article). An inclusion rate of 90% for these four factors was set 
for sample size determination. This high but not perfect inclusion rate was meant to account for 
unintentional errors while aiming for an achievable high reporting integrity and transparency standard. A 
90% target inclusion rate was considered reasonable given that since 2017, PubMed has included COIs below 
the abstract when supplied by the publisher [5]. The null hypothesis was that each variable would be present 
in at least 90% of the news reports, and the alternative hypothesis was that each variable would be present in 
less than 90% of the news reports. 


Medical articles for evaluation were obtained from the PubMed database using the following strategy: 
((((("The New England Journal of Medicine" [Journal]) OR ("JAMA"[Journal])) OR ("BMJ (Clinical Research ed.)" 
[Journal])) OR (‘Annals of Internal Medicine" [Journal])) OR ("Lancet (London, England)" [Journal])) OR 
("Nature medicine" [Journal]) AND ffrft[Filter] AND randomizedcontrolledtrial[Filter] 


Journals were restricted to the New England Journal of Medicine, JAMA, BMJ, the Annals of Internal Medicine, 
Lancet, or Nature Medicine. These were chosen due to their high profile and thus increased likelihood of 
being covered by the news media. 


Results were filtered to only include articles with free full-text availability. Also the results were filtered to 
only include randomized controlled trials. 


The sample size was determined using several methods: (a) a power analysis was conducted using a one- 
sample test of proportions. For a 99% level of confidence and an expected proportion of 90% with a margin 
of error of 15%, a sample size of 27 would be required [6]; (b) to detect an effect size of 15% or greater from 
the null hypothesis of 90% using a two-sided test, an alpha of 0.05, a power of 80%, a sample size of 41 
would be required [7,8]; and (c) finally, a common rule of thumb for sample size determination is to have 5 to 
10 samples per variable [9]. Given these results, it was determined that a sample size of 50 would be 
reasonable and appropriate. 


Statistical analyses using IBM SPSS Statistics for Windows, Version 29 (Released 2023; IBM Corp., Armonk, 
New York) employed a binomial test of the null hypothesis. Given that an inclusion rate of over 90% would 
support the null hypothesis, a one-tailed p-value was utilized to determine if the inclusion rate for each 
variable was under 90%. The Bonferroni correction, which adjusts the significance level for multiple 
comparisons to reduce the likelihood of Type I errors, was used to determine the significance of p-values 
[10]. The SAMPL guidelines for health research reporting were followed [11]. 


Recognizing the potential sensitivity of the data and the importance of unbiased analysis, we anonymized 
the identities of the research articles and the news organizations. This approach was fundamental not only 
to prevent any undue focus on specific entities but also to maintain an objective stance that fosters a 
constructive dialogue about improving medical research reporting without targeting any specific 
organization. By anonymizing the data, we aim to highlight systemic trends and opportunities for 
enhancement rather than attributing shortcomings to individual publishers. This ethical consideration 
underscores our commitment to a respectful and responsible examination of the media landscape, 
contributing to a broader understanding and trust in medical science communication. 


The dataset supporting our findings is accessible on the Zenodo repository, which offers transparency and 
enables further research in this area [12]. 


Results 


A total of 50 clinical research trials and their corresponding news reports were analyzed. The 50 news 
reports came from a total of 34 unique news organizations. The clinical research trials were all from 2023- 
2024. 


Conflicts of interest (COIs) were mentioned in 10 articles (20% + 5.8%, p < 0.001), indicating a significant 
underreporting compared to the 90% threshold. Study limitations were acknowledged in 14 articles (28% + 
6.5%, p < 0.001), also significantly lower than the 90% threshold. Inferential statistics, such as p-values or 
confidence intervals with effect sizes, were included in 19 articles (38% + 7.0%, p < 0.001), again falling short 
of the 90% threshold. The news report agreed with the medical trial’s findings in 43 out of 50 cases (86% + 
5.0%), which is not significantly different from the null hypothesis of 90% (p = 0.240). However, if a higher 
standard of 95% agreement were set, then this rate of 86% would be determined to be significantly lower (p = 
0.009) (Table 1). 
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Reporting Criteria 
COls 

Study limitations 
Inferential statistics 


Aligned conclusions 


Number of Reports (%) 95% Confidence Interval p-Value 
10 (20%) 14.2 - 25.8% < 0.001 
14 (28%) 21.5 - 34.5% < 0.001 
19 (38%) 31.0 - 45.0% < 0.001 
43 (86%) 81.0 - 91.0% 0.230 


TABLE 1: The inclusion rate of reporting criteria in news reports 


In all seven cases where the conclusions did not match, the reason was overhyping results by making 
unqualified claims not found in the research trial. The specific reasons for a mismatch of conclusions were as 
follows. (1) One news report only mentioned part of the conclusion indicating positive results and did not 
mention that the results were negative in the other arm of the trial. (2) Another news report expanded the 
finding of decreased resource utilization to include that patients received improved care when this was not 
proven and not one of the study’s outcomes. (3) A third mismatch occurred when the medical article showed 
findings restricted to an age group, but the news report indicated it was applicable to all age groups. (4) A 
fourth news report agreed with the article’s conclusion that a supplement “might” reduce cardiovascular 
events. Still, the report's title stated unequivocally that the supplement reduced cardiovascular events. 

(5) One news report stated that the new intervention completely eliminated the need for maintenance 
medical therapy. In contrast, the research determined that the intervention reduced but did not eliminate 
the need for ongoing maintenance medical therapy. (6) One medical research article found only that 
recurrence-free survival was improved with the new therapy; the news report stated that not only was 
recurrence-free survival improved, but there was also a reduction in deaths. (7) The final discordant study 
was due to the news report stating that the increased number of deaths in the intervention arm was unlikely 
due to the intervention. The research article, however, was much more cautious, stating that ongoing 
research into this finding was necessary. 


Only four news reports included all four reporting criteria, and three did not include any of the reporting 
criteria analyzed. 


Discussion 


This study reveals significant gaps in reporting critical contextual information in online medical news 
articles, such as COIs, study limitations, and inferential statistics. While research conclusions were generally 
conveyed accurately, underreporting these key elements raises concerns about the transparency and 
credibility of medical research communication. When news reports disagreed with the medical research, it 
was invariably due to overstating research findings by the news reports. 


These findings have important societal implications. The low rates of reporting COIs, study limitations, and 
inferential statistics decrease readers’ ability to properly understand and evaluate medical research. Without 
access to this contextual information, readers may overestimate the significance or applicability of research 
findings, leading to potential misinterpretations and the spread of misinformation [13]. Incomplete 
reporting of medical research can also have serious ramifications for public health outcomes. Omitting 
relevant information, such as COIs, hinders readers’ ability to gauge a study's possible weaknesses, given 
that COIs have been shown to affect study results [14]. Additionally, the lack of inclusion of statistical 
analyses and a discussion of study limitations can prevent readers from fully evaluating the significance and 
generalizability of the research findings. 


This study reinforces findings from previous research. One study reporting on 500 health news stories from 
major US news organizations found that most of these stories failed to adequately address costs, harms, 
benefits, the quality of the evidence, and alternative options [15]. Another study found that "spin" in press 
releases and news reports was related to the presence of "spin" in the abstract of peer-reviewed articles of 
randomized controlled trials (RCTs). This "spin" led to an overestimation of the benefit of the experimental 
treatment in 27% of reports based on the press release and 24% of reports based on the news item, compared 
to the full-text peer-reviewed article [16]. Similarly, another study evaluated 1889 health news stories and 
found that most stories failed to adequately address costs, harms, benefits, evidence quality, and alternative 
options when covering healthcare products and procedures [13]. Another study looking at pharmaceutical 
research found significant shortcomings, with 53% of news reports failing to mention potential harms and 
70% not mentioning costs. This study also found that the COIs of experts quoted in news reports were 
disclosed only 39% of the time [17]. 


These findings underscore the need for more comprehensive and balanced reporting of medical research by 
the news media, as inadequate coverage can lead to misinformed healthcare consumers and decision- 
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makers. However, efforts to combat misinformation through individualized solutions, such as labeling fake 
news sources and fact-checking, have significant limitations. They can exacerbate the problem of 
misinformation by failing to address the underlying systemic issues in journalism, tech platform dominance, 
and governmental policies [18]. 


To address these challenges, one potential solution would be the adoption of structured reporting formats 
for medical news articles, similar to those utilized in a standard clinical trial abstract and manuscript. Such 
structured reporting has been shown to improve the communication of medical information. For example, a 
study of 330 radiology reports found that referring physicians were more satisfied with structured reports 
and that the structured reports had greater clarity [19]. While not a uniform requirement, many imaging 
societies recommend structured reporting of scan results, including the American Society of Nuclear 
Cardiology [20] and the American College of Radiology [21]. Similarly, a structured format for medical 
research articles is recommended. For example, the International Committee of Medical Journal Editors 
recommends structured abstracts and a standardized overall structure for medical research articles [22]. 


One recommendation for news reporting is to follow standardized inclusion criteria when reporting on 
medical research. While news reports may utilize a narrative style, the contents would still follow a 
standardized flow. Specifically, the establishment of best practices in medical news reporting might call for 
the routine inclusion of background information, study methods, main results with statistical analyses, a 
discussion of implications and limitations, a conclusion that accurately reflects the research findings, and 
finally, a disclosure of COIs. 


While our study provides valuable insights, it is essential to acknowledge its limitations. Our focus on major 
medical journals and English-language publications may not fully represent the global landscape of medical 
news reporting. This study only looked at randomized clinical trials published in major medical journals in 
2023 and 2024. Future research should expand the scope of analysis to include a broader array of sources 
and languages, and a wider variety of research types, providing a more comprehensive understanding of the 
challenges and opportunities for improving medical research communication worldwide. Despite these 
limitations, this study makes significant contributions to evaluating online misinformation. By focusing on 
clear and quantifiable components, medical news reporting was objectively assessed, and key areas for 
improvement were identified. Furthermore, the findings provide a compelling case for adopting 
standardized reporting to improve online communication of medical research. These findings also lay the 
groundwork for future research and interventions to promote accurate, transparent, and trustworthy 
communication of scientific knowledge to the public. 


Conclusions 


This study reveals significant gaps in the reporting of medical research by online news organizations, 
particularly regarding transparency about COIs, discussion of study limitations, and the inclusion of relevant 
statistical data. Despite the encouraging agreement on research conclusions, there is a pressing need for 
improved reporting standards. Implementing structured reporting formats, similar to those used in clinical 
trial abstracts and manuscripts could enhance the quality and transparency of medical research 
communication. Collaboration among journalists, news organizations, and medical researchers is crucial to 
achieve this. By working together to establish and promote best practices, these stakeholders can foster a 
more informed public discourse on health and science topics, ultimately contributing to better health 
outcomes and decision-making. 
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